A heterogeneous distributed visual servoing system for real-time robotic assembly applications

We describe a multi-loop robotic control architecture that for the fine-motion control required for object tracking uses two different loops that depend on entirely different approaches for visual servoing. One tracking loop uses fast blob analysis of the camera images for servo control. The other tracking loop, which is more robust but slower, uses model-based line matchings for a more robust calculation of instantaneous pose. A control arbitrator gives a higher priority to the faster blob-analysis based control loop. But should any disturbances in the system cause the loss of servo control with respect to the input received from the blob-analysis based control loop, the control arbitrator automatically switches to the other more robust control loop. As a result, the system shows high level of fault tolerance under non-cooperative conditions, such as severe occlusion and lighting condition changes.